Hyperspectral Image Classificatio Based O a Fast Bregma Sparse Multi Omial Logistic Regressio Algorithm

The Sparse Multinomial Logistic Regression (SMLR) method introduced in (Krishnapuram, 2005) is among the state-of-the-art in supervised learning. However its application to large datasets, such as hyperspectral imagery is still a rather challenging task from the computational point of view, sometimes even impossible to perform. In this paper, the Bregman iteration-based SMLR method (Bregman-SMLR) recently introduced in (Bioucas-Dias, 2008) is applied to hyperspectral data classification problems. The Bregman method allows replacing a difficult, non-smooth convex problem with a sequence of quadratic plus diagonal l2-l1 problems which are very easy to solve (Bioucas-Dias, 2008). Compared with the SMLR algorithm, the reduction of computational complexity is on the order of d(m-1) 3 (d is the number of features, and m is the number of classes.) The effectiveness of the proposed method is evaluated with simulated data sets and a real AVIRIS image. Results are presented and compared with others obtained by state-of-the-art supervised algorithms. * Corresponding author. This work was supported by Marie Curie Grant MEST-CT-2005-021175 from the European Commission. 1. I TRODUCTIO The sparse multinomial logistic regression (SMLR) method introduced in (Krishnapuram, 2005) is among the state-of-the-art in supervised learning. The core of the SMLR is the solution of a two-term optimization problem: one term is the logistic regression and the other is a Laplacian prior which enforces sparseness, thus controlling the machine complexity. However, the SMLR application to large datasets, such as hyperspectral imagery, is still a rather challenging task from the computational point of view, being sometimes even impossible to perform. This is because SMLR has the complexity of the iterative reweighted least squares (IRLS) algorithm for maximum likelihood estimation of feature weights. To lighten the SMLR computational burden, a fast sparse multinomial logistic regression (FSMLR) was introduced in (Borges, 2006) to implement an iterative scheme (based on the block Gauss-Seidel method) to compute the feature weights of the decision function. The computational gain with respect to the SMLR algorithm is of the order of the number of classes. The FSMLR algorithm is thus well-suited to hyperspectral data sets with a large number of classes. However, when dealing with classification problems with large training sets resulting, for example, from kernel-based regression, the FSMLR method is still very complex in computational terms. In this paper, the Bregman iteration-based SMLR method (Bregman-SMLR) recently introduced in (Bioucas-Dias, 2008) is applied to hyperspectral data classification problems. The Bregman method allows replacing a difficult, non-smooth convex problem with a sequence of quadratic plus diagonal l2-l1 problems which are very easy to solve. If d is the number of features and m is the number of classes, the complexity of the Bregman-SMLR method is O(d 2 ), which is in contrast with the O((d(m1)) 3 ) figure of SMLR. As a result, the reduction of computational complexity is on the order of d(m-1) 3 . In order to illustrate the effectiveness of the Bregman-SMLR method, we apply it to simulated data sets and real AVIRIS hyperspectral image and compare the obtained results with those provided by the FSMLR, the support vector machines (SVMs), and the linear discriminant analysis (LDA) (Camps-Valls, 2005) in terms of the following aspects: 1) overall accuracy; 2) computational cost; 3) robustness to noise; and 4) number of the training samples required.